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Abstract. In this paper, we present a method based on belief functions 
to evaluate the quality of the optimal assignment solution of a classical 
association problem encountered in multiple target tracking applications. 
The purpose of this work is not to provide a new algorithm for solving 
the assignment problem, but a solution to estimate the quality of the 
individual associations (pairings) given in the optimal assignment solu- 
tion. To the knowledge of authors, this problem has not been addressed 
so far in the literature and its solution may have practical aspects for 
improving the performances of multisensor-multitarget tracking systems. 
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1 Introduction 

Efficient algorithms for modern multisensor- multitarget tracking (MS-MTT) sys- 
tems [1, 2] require to estimate and predict the states (position, velocity, etc) of the 
targets evolving in the surveillance area covered by the sensors. The estimations 
and the predictions are based on sensors measurements and dynamical models 
assumptions. In the monosensor context, MTT requires to solve the data asso- 
ciation (DA) problem to associate the available measurements at a given time 
with the predicted states of the targets to update their tracks using filtering 
techniques (Kalman filter, Particle filter, etc). In the multisensor MTT context, 
we need to solve more difficult multi-dimensional assignment problems under 
constraints. Fortunately, efficient algorithms have been developed in operational 
research and tracking communities for formalizing and solving these optimal as- 
signments problems. Several approaches based on different models can be used 
to establish rewards matrix, either based on the probabilistic framework [1,3], 
or on the belief function (BF) framework [4-7]. In this paper, we do not focus on 
the construction of the rewards matrix 1 , and our purpose is to provide a method 
to evaluate the quality (interpreted as a confidence score) of each association 
(pairing) provided in the optimal solution based on its consistency (stability) 
with respect to all the second best solutions. 

1 We assume that the rewards matrix is known and has been obtained by a method 
chosen by the user, either in the probabilistic or in the BF framework. 
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The simple DA problem under concern can be formulated as follows. We have 
m > 1 targets T) (i = 1, ... ,m), and n > 1 measurements 2 Zj (j = 1, . . . , n) 
at a given time k , and a m x n rewards (gain/payoff) matrix 1? = Km)] 
whose elements > 0 represent the payoff (usually homogeneous to the 

likelihood) of the association of target Tj with measurement Zj , denoted (Xi, z^). 
The data association problem consists in finding the global optimal assignment 
of the targets to some measurements by maximizing 3 the overall gain in such a 
way that no more than one target is assigned to a measurement, and reciprocally. 

Without loss of generality, we can assume > 0 because if some elements 

of Si were negative, we can always add the same maximal negative value 
to all elements of Si to work with a new payoff matrix Si ' = having all 

elements > 0, and we get the same optimal assignment solution with Si 

and with Si' . Moreover, we can also assume, without loss of generality m < n, 
because otherwise we can always swap the roles of targets and measurements in 
the mathematical problem definition by working directly with Si 1 instead, where 
the superscript t denotes the transposition of the matrix. The optimal assignment 
problem consists of finding the m x n binary association matrix A = [a(i, j)] 
which maximize the global rewards R(Si, A) given by 

m n 

R(S2, A) = K, j ) (1) 

i=l j = 1 

fE”=i a(i,j) = l (* = 1, . . . , m) 

Subject to | J2Z i «(*> 3) < 1 U = b • ■ ■ > n) (2) 

[a(i, j) € {0, 1} (i = 1,... ,m and j = 1, . ,.,n) 

The association indicator value a(i,j) = 1 means that the corresponding 
target T,; and measurement Zj are associated, and a(i,j) = 0 means that they 
are not associated (i = 1, . . . , m and j = 1, . . . , n). 

The solution of the optimal assignment problem stated in (l)-(2) is well 
reported in the literature and several efficient methods have been developed in 
the operational research and tracking communities to solve it. The most well- 
known algorithms are Kuhn-Munkres (a.k.as Hungarian) algorithm [8,9] and its 
extension to rectangular matrices proposed by Bourgeois and Lassalle in [10], 
Jonker-Volgenant method [11], and Auction [12]. More sophisticated methods 
using Murty’s method [13], and some variants [3, 14-19], are also able to provide 
not only the best assignment, but also the m-best assignments. We will not 
present in details all these classical methods because they have been already 
well reported in the literature [20, 21], and they are quite easily accessible on the 

2 In a multi-sensor context targets can be replaced by tracks provided by a given 
tracker associated with a type of sensor, and measurements can be replaced by 
another tracks set. In different contexts, possible equivalents are assigning personnel 
to jobs or assigning delivery trucks to locations. 

3 In some problems, Si = [cj(i, j)] represents a cost matrix whose elements are the 
negative log-likelihood of association hypotheses. In this case, the data association 
problems consists in finding the best assignment that minimizes the overall cost. 
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web. In this paper, we want to provide a confidence level (i.e. a quality indicator) 
in the optimal data association solution. More precisely, we are searching an 
answer to the question: how to measure the quality of the pairings a(i,j ) = 1 
provided in the optimal assignment solution A? The necessity to establish a 
quality indicator is motivated by the following three main reasons: 



1. In some practical tracking environment with the presence of clutter, some 
association decisions ( a(i,j ) = 1) are doubtful. For these unreliable associ- 
ations, it is better to wait for new information (measurements) instead of 
applying the hard data association decision, and making potentially serious 
association mistakes. 

2. In some multisensor systems, it can be also important to save energy con- 
sumption for preserving a high autonomy capacities of the system. For this 
goal, only the most trustful specific associations provided in the optimal 
assignment have to be selected and used instead of all of them. 

3. The best optimal assignment solution is not necessarily unique. In such sit- 
uation, the establishment of quality indicators may help in selecting one 
particular optimal assignment solution among multiple possible choices. 

Before presenting our solution in Section 2, one must recall that the best, as well 
as the 2nd-best, optimal assignment solutions are unfortunately not necessarily 
unique. Therefore, we must also take into account the possible multiplicity of 
assignments in the analysis of the problem. The multiplicity index of the best 
optimal assignment solution is denoted f3 1 > 1, and the multiplicity index of the 
2nd-best optimal assignment solution is denoted fa > 1 , and we will denote the 
sets of corresponding assignment matrices by A\ = {A^ , k± = 1 . . . ,fa} and 

by Ai = {A^, = 1 . . . , fa}. The next simple example illustrates a case with 

multiplicity of 2nd-best assignment solutions for the reward matrix 17 1 . 

Example: fa = 1 and fa = 4 (i.e. no multiplicity of Ai and multiplicity of A 2 ) 



17 1 



' 1 11 45 30' 
17 8 38 27 
10 14 35 20 



This reward matrix provides a unique best assignment Ai providing R\ (17i , Ai ) = 
86, and fa = 4 second-best assignment solutions providing A^ 2 ) = 82 

(fc 2 = 1,2, 3, 4) given by 



Ai 



'0 0 1 O' 
0 0 0 1 
0 10 0 





'00 0 1 




'0 0 10' 




'0 0 10' 




'0 0 0 r 


A ^2 — 1 — 
^2 — 


0 0 10 
0 10 0. 


\k 2 =2 _ 


10 0 0 
0 0 0 1. 


\k 2 =S _ 

i -^2 


0 0 0 1 
.1 0 0 0. 


A fc 2=4 _ 
1 -^-2 


1000 

.0010. 
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2 Quality of the Associations of the Optimal Assignment 

To establish the quality of the specific associations (pairings) (i,j) satisfying 
a\ (i,j) = 1 belonging to the optimal assignment matrix Ai, we propose to use 
both Ai and 2nd-best assignment solution A 2 . The basic idea is to compare the 
values ai(i,j) with a, 2 (i,j) obtained in the best and in the 2nd-best assignments 
to identify the change (if any) of the optimal pairing (*, j). Our quality indicator 
will depend on both the stability of the pairing and its relative impact in the 
global reward. The proposed method works also when the 2nd-best assignment 
solution A 2 is not unique (as in our example). The proposed method will also 
help to select the best (most trustful) optimal assignment in case of multiplicity 
of Ai matrices. 

2.1 A Simplistic Method (Method I) 

Before presenting our sophisticate method based on belief functions, let’s first 
present a simplistic intuitive method (called Method I). For this, let’s assume at 
first that Ai and A 2 are unique (no multiplicity occurs). The simplistic method 
uses only the ratio of global rewards p = f?2(f2, A 2 )/f?i(17, Ai) to measure the 
level of uncertainty in the change (if any) of pairing (i, j) provided in Ai and A 2 . 
More precisely, the quality (trustfulness) of pairings in an optimal assignment 
solution Ai, denoted 4 qi(i,j), is simply defined as follows for i = 1 ,m and 
j = 2,...,n: 

{ 1, if ai(i,j) + a 2 {i, j) = 0 

1-P if ai(i, j) + a 2 (i, j) = 1 (3) 

1, if oi(i,i) + o 2 (i,j) = 2 

By adopting such dehnition, one commits the full confidence to the compo- 
nents (i,j) of Ai and A 2 that perfectly match, and a lower confidence value (a 
lower quality) of 1 — p to those that do not match. To take into account the 
eventual multiplicities (when 02 > 1) of the 2nd-best assignment solutions A?) 2 , 
fc 2 = 1,2,..., 02, we need to combine the Q/(Ai, A]] 2 ) values. Several methods 
can be used for this, in particular we can use either: 

— A weighted averaging approach: The quality indicator component qi{i,j ) 
is then obtained by averaging the qualities obtained from each comparison 
of Ai with A^ 2 . More precisely, one will take: 

P2 

qi(i,j) - J2 w ( A 2 2 )9/ 2 (*>j) ( 4 ) 

& 2=1 

where qj 2 {i,j) is defined as in (3) (with 02 ( 4 , j) replaced by in 

the formula), and where w{ A* 2 ) is a weighting factor in [0,1], such that 
i*^ 2 ) = 1. Since ah assignments A (j 2 have the same global reward 
value i? 2 , then we suggest to take w( Aj 2 ) = 1 / 02 - A more elaborate method 

4 The subscript I in notation refers to Method I. 
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would consist to use the quality indicator of A (j 2 based on the 3rd-best 
solution, which can be itself computed from the quality of the 3rd assignment 
solution based on the 4th-best solution, and so on by a similar mechanism. 
We however don’t give more details on this due to space constraints. 



— A belief-based approach (see [22] for basics on belief functions): A second 
method would express the quality by a belief interval [g™^, j), j)] 

in [0,1] instead of single real number qi(i,j) in [0,1]. More precisely, one 
can compute the belief and plausibility bounds of the quality by taking 
j)) = min k 2 q^ 2 (i,j) and q^ ax {i,j) = Pl{ar{i,j)) = 
ma x k2 qj 2 {i,j), with qj 2 {i,j) given by (3) and a 2 (i,j) replaced by a% a (i,j) 
in the formula. Hence for each association one can define a basic 

belief assignment (BBA) rriij{.) on the frame of discernment 0 = {T = 
trustful, -i T = not trustful}, which will characterize the quality of the pairing 
(i,j) in the optimal assignment solution Ai, as follows: 

\m ij {T)=qT%i,j) 

) mij (-,T) = l-qr x (i,j) (5) 

[ rriij (T U ~>T) = qf^j) - qf n (i,j) 



Remark: In practice, only the pairings 5 (■ i,j ) such that = 1 are use- 

ful in tracking algorithms to update the tracks. Therefore, we don’t need to 
pay attention (compute and store) the qualities of components (i,j) such that 
ai (i,j) = 0. 

2.2 A More Sophisticate and Efficient Method (Method II) 



The previous method can be easily applied in practice but it does not work very 
well because the quality indicator depends only on the p factor, which means that 
all mismatches between the best assignment Ai and the 2nd-best assignment 
solution A 2 have their quality impacted in the same manner (they are all taken 
as 1 — p). As a simple example, if we consider the rewards matrix fl\ given in 
our example, we will have p = R 2 (f2i, A 2 2 )/f?i(l?i, Ai) = 82/86 ~ 0.95, and 
we will get using method I with the weighting averaging approach (using same 
u/Aj 2 ) = l/fo = 0.25 for k 2 = 1, 2, 3, 4) the following quality indicator matrix: 



Q/(Ai, A 2 ) 



1 

02 



02 

J2 Qr(Ai,A* 2 ) 



k 2 = 1 



1.0000 1.0000 0.5233 0.5233 
0.5233 1.0000 0.7616 0.2849 
0.7616 0.28 4 9 0.7616 0.7616 



(6) 



We observe that optimal pairings (2,4) and (3,2) get the same quality value 
0.2849 with the method I (based on averaging), even if these pairings have dif- 
ferent impacts in the global reward value, which is abnormal. If we use the 
method I with the belief interval measure based on (5), the situation is worst 
because the three optimal pairings (1,3), (2,4) and (3,2) will get exactly same 
belief interval values [0.0465,1]. To take into account, and in a better way, the 

5 given in the optimal solution found for example with Murty’s algorithm. 
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reward values of each specific association given in the best assignment Ai and 
in the 2nd-best assignment A 2 2 , we propose to use the following construction of 
quality indicators depending on the type of matching (called Method II): 

— When a-i (i, j) = a 2 2 {i,j ) = 0, one has full agreement on “non-association” 

( Ti,Zj ) in Ai and in A 2 2 and this non-association ( Ti,Zj ) has no impact on 
the global rewards values Ri(f2, Ai) and R 2 {fl, A?] 2 ), and it will be useless. 
Therefore, we can set its quality arbitrarily to = 1. 

— When ai(i,j) = a 2 2 (i,j ) = one has a full agreement on the association 
(Tj, Zj) in Ai and in A$ 2 and this association (T*, Zj) has different impacts in 
the global rewards values Ri(f2, Ai) and R 2 (f2, A^ 2 ). To qualify the quality 
of this matching association (T), Zj), we define the two BBA’s on X = ( T, , Zj ) 
and X U -A (the ignorance), for s = 1,2: 



f m s (A) = a s (i,j) ■ u(i,j)/R e ({2, A„) 
|m s (A'UnX) = 1 — m s (A) 

Applying the conjunctive rule of fusion, we get 



m(X) = mi(X)m 2 (X) + m i(X)m 2 (X U ~<X) + mi (A U X)m 2 (X ) 
m(X U -<X) = mi (A U ^A)m 2 (A U -.A) 



Applying the pignistic transformation 6 [24], we get finally BetP(X) = m(X)+ 
1 • m ( X U -iA) and BetP(-> X) = i • m ( X U ->A). Therefore, we choose the 
quality indicator as qjj(i,j) = BetP(X). 

— When = 1 and a 2 2 (i,j) = 0, one has a disagreement (conflict) on 

the association ( Ti,Zj ) in Ai and in (Ti,Zj 2 ) in A^ 2 , where j 2 is the mea- 
surement index such that a 2 (i,j 2 ) = 1. To qualify the quality of this non- 
matching association (T), Zj), we define the two following basic belief assign- 
ments (BBA’s) of the propositions A = (T), Zj) and Y = (: Tj , Zj 2 ) 



mi (A) = ai (i,j) ■ 
mi(AUf) = l- 



fliin.Ai) 



mi (A) 



I m a (y) = a 2 (i,j 2 ) 



and ( ' R 2 (n,A k 2 z 

ms(AUY) = l-m 2 (Y) 



(9) 



Applying the conjunctive rule, we get m( A flK = 0) = mi(A)m 2 (Y) and 
f m(A) = rrii(X)m 2 (A U Y) 

m(Y)=mi(AUY)m 2 (Y) (10) 

[m(A U Y) = mi (A U Y)m 2 (A U Y) 

Because we need to work with a normalized combined BBA, we can choose 
different rules of combination (Dempster-Shafer’s, Dubois-Prade’s, Yager’s 

We have chosen here BetP for its simplicity and because it is widely known, but 
DSmP could be used instead for expecting better performances [23]. 
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rule [23] , etc) . In this work, we recommend the Proportional Conflict Redis- 
tribution rule no. 6 (PCR6), proposed originally in DSmT framework [23], 
because it has been proved very efficient in practice. So, we get with PCR6: 

f m(X) = mi (X)m 2 (X U Y) + rm(X) ■ "&£££) 

\ m(Y) = mi (X U Y)m 2 (Y) + m 2 (X) ■ ( n ) 

[m(X U Y) = mi(X U Y)m 2 (X U Y) 



Applying the pignistic transformation, we get finally BetP(X) = m(X) + ^ • 
m(X U Y) and BetPiY) = m(Y ) + i • m(X U Y). Therefore, we choose the 
quality indicators as follows: qfj (b j) = BetP(X), and q/j (i, j 2 ) = BetP(Y). 

The absolute quality factor Q a bs{ Ai, Aj 2 ) of the optimal assignment given in 
Ai conditioned by A 2 2 , for any k 2 £ { 1 , 2, . . . , /3 2 } is defined as 

m n 

Q abs ( Ai,A 2 2 ) =y^y~]ai (i,j)qii(i,j) (12) 

i= 1 j = l 

Example (continued): If we apply the Method II (using PCR6 fusion rule) to 
the rewards matrix fl 1, then we will get the following quality matrix (using 
weighted averaging approach) 



Q//(Ai,A 2 ) 



1 

- Y, Q//(Ai, A2 2 ) 



T.0000 1.0000 0.7440 0.7022' 
0.7200 1.0000 0.8972 0.5753 
0.8695 0.4957 0.9119 0.8861 



with the absolute quality factors Q a b s (Ai,A.2 2 1 ) ~ 1.66, Qabsi^-i--^' 2 ) ~ 
1.91, Q abs (Ai, Aj 2 "' 5 ) w 2.19, Q abs (Ai, A h 2 2= B ~ 1.51. Naturally, we get 

Q abs (A!, A^ 2 = 3 ) > Q abs (Ai, A^ 2 = 2 ) > Q a fes(A 1 , A2 2=1 ) > Q^A,, A^ 2 = 4 ) 

because Ai has more matching pairings with A 2 2 ~ 3 than with other 2nd-best 
assignment A 2 2 (k 2 ^ 3), and those pairings have also the strongest impacts in 
the global reward value. One sees that the quality matrix Q 77 differentiates the 
qualities of each pairing in the optimal assignment Ai as expected (contrari- 
wise to Method I). Clearly, with Method I we obtain the same quality indicator 
value 0.2849 for the specific associations (2,4) and (3,2) which seems intuitively 
not very reasonable because the specific rewards of these associations impact 
differently the global rewards result. If the method II based on the belief in- 
terval measure computed from (5) is preferred 7 , we will get respectively for the 
three optimal pairings (1,3), (2,4) and (3,2) the three distinct belief interval 
[0.5956,0.8924], [0.4113,0.7699] and [0.3524,0.6529]. These belief intervals show 
that the ordering of quality of optimal pairings (based either on the lower bound, 
or on the upper bound of belief interval) is consistent with the ordering of qual- 
ity of optimal pairings in Q//(Ai, A 2 ) computed with the averaging approach. 
Method II provides a better effective and comprehensive solution to estimate the 
quality of each specific association provided in the optimal assignment solution 
Aj. 

7 just in case of multiplicity of second best assignments. 
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3 Conclusion 

In this paper we have proposed a method based on belief functions for estab- 
lishing the quality of pairings belonging to the optimal data association (or as- 
signment) solution provided by a chosen algorithm. Our method is independent 
of the choice of the algorithm used in finding the optimal assignment solution, 
and, in case of multiple optimal solutions, it provides also a way to select the 
best optimal assignment solution (the one having the highest absolute quality 
factor). The method developed in this paper is general in the sense that it can be 
applied to different types of association problems corresponding to different sets 
of constraints. This method can be extended to SD-assignment problems. The 
application of this approach in a realistic multi-target tracking context is under 
investigations and will be reported in a forthcoming publication if possible. 
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